Learning Graph Structure With Parametric and Non-Parametric Models

In discrete undirected graphical models, the conditional independence of the node labels Y is specified by the graph structure. We study the case where there is another input random vector X (e.g. observed features) such that the distribution P (Y | X) is determined by functions of X that characterize the (higher-order) interactions among the Y ’s. The main contribution is to learn the graph structure and the functions conditioned on X at the same time. Parameterizing the graphical models with potential functions might lead to overparameterization. We prove that the discrete undirected graphical models with feature X are equivalent to the multivariate discrete models. The reparameterization of the potential functions in graphical models by conditional log odds ratios of the latter offers advantages in the representation of the conditional independence structure. And the two parameterizations are proved to be equivalent. In addition, the spaces of conditional log odds ratios can be chosen flexibly. They could be linear functional spaces (parametric), or separable Reproducing Kernel Hilbert Spaces determined by kernels (non-parametric). To obtain a sparse estimation of the graph structure, we impose a Structure Lasso (SLasso) penalty on groups of the conditional log odds ratios to learn the graph structure. These groups with overlaps are designed to enforce hierarchical function selection. An efficient gradient descent algorithm is given to estimate the complete model. The global convergence of the algorithm is guaranteed. And a greedy approach is applied when the graph is large. The BGACV tuning method is derived to select the tuning parameter. It achieves satisfactory numerical results in simulation studies. x The asymptotic analysis shows that the SLasso method is consistent in terms of estimating the graph structure. The consistency properties hold for both the parametric models and the nonparametric models. The experiments show that the SLasso method is able to recover the graph structure with increasing sample size. It also outperforms other methods in the simulation studies.